Watermark detection

ABSTRACT

An improved method and arrangement for detecting a watermark in an information (e.g. image or video) signal are disclosed. The detection is more reliable and less vulnerable to image processing by subjecting the suspect image (q) and the watermark to be detected (w) to Symmetrical Phase Only Matched Filtering ( 24 - 28,16 ) prior to detecting ( 29 ) the amount of correlation (d) between said signals.

FIELD OF THE INVENTION

[0001] The invention relates to a method of detecting a watermark in aninformation signal that has possibly been watermarked by modifyingvalues of said information signal in accordance with (temporally orspatially) corresponding values of a watermark pattern. The inventionalso relates to an arrangement for detecting a watermark.

BACKGROUND OF THE INVENTION

[0002] A prior art method as defined in the opening paragraph isdisclosed in International Patent Application WO-A-98/03014. Thewatermark is detected by computing the correlation of the suspectinformation signal with an applied watermark pattern, and comparing thecorrelation with a predetermined threshold. If the correlation is largerthan the threshold, the watermark is said to be present, otherwise it issaid to be absent. The larger the correlation is, the more reliable thedetection is and the more processing is allowed until the watermark cannot be detected anymore. As disclosed in WO-A-98/03014, the informationsignal and/or the watermark pattern may be subjected to matchedfiltering before the correlation is computed. The reliability of thewatermark detection is drastically improved thereby.

[0003] A problem of the prior art watermark detection method is that the(temporal or spatial) position of the watermark pattern with respect tothe information signal is not absolutely known. If the position of thewatermark applied to the detector differs from the location duringembedding, the correlation will be small and the watermark canerroneously detected to be absent.

OBJECT AND SUMMARY OF THE INVENTION

[0004] It is an object of the invention to further improve the method ofdetecting a watermark.

[0005] This is achieved by the method defined in claim 1 and thearrangement defined in claim 6. Advantageous embodiments are defined inthe subclaims.

[0006] The invention exploits the insight that the correlation of theinformation signal and the applied watermark for a number of possiblepositions of the watermark is best computed in the Fourier domain, andthat the robustness and reliability of detection can be improved byapplying Symmetrical Phase Only Matched Filtering (SPOMF) to theinformation signal and the watermark before correlation. SPOMF, which isknown per se in the field of pattern recognition, postulates that mostof the relevant information needed for correlation detection is carriedby the phase of Fourier coefficients. In accordance herewith, themagnitudes of the complex Fourier coefficients are normalized to havesubstantially the same magnitudes.

[0007] Although the adjective “symmetrical” in the expression SPOMFrefers to filtering both the information signal and the watermark, theinventors have found that normalizing the Fourier transformedinformation signal contributes most to the improvement of thecorrelation detection. Also normalizing the magnitudes of the Fouriertransformed watermark constitutes an embodiment of the invention.

[0008] The step of normalizing the magnitude of complex Fouriercoefficients includes dividing each Fourier coefficient by its absolutevalue. However, the inventors have found that the detection strengthdoes not decrease significantly if each coefficient is divided by itsreal or imaginary part, depending on which is the largest. This yields acourser approximation for the normalization (the magnitudes will varybetween 1 and {square root}{square root over (2)}), but reduces thenumber of calculations considerably.

BRIEF DESCRIPTION OF THE DRAWINGS

[0009]FIG. 1 shows schematically an arrangement for embedding awatermark in a signal.

[0010]FIGS. 2 and 3 show diagrams to illustrate the operation of theembedder which is shown in FIG. 1.

[0011]FIG. 4 shows schematically an arrangement for detecting theembedded watermark.

[0012]FIGS. 5, 6A and 6B show diagrams to illustrate the operation ofthe detector which is shown in FIG. 4.

[0013]FIG. 7 shows a device for playing back a video bit stream with anembedded watermark.

[0014]FIG. 8 shows schematically an embodiment of the arrangement fordetecting the embedded watermark in accordance with the invention.

[0015]FIGS. 9A and 9B show diagrams to illustrate the operation of thedetector which is shown in FIG. 8.

[0016]FIG. 10 shows schematically a further embodiment of thearrangement for detecting the embedded watermark in accordance with theinvention.

DESCRIPTION OF PREFERRED EMBODIMENTS

[0017] For the sake of convenience, the watermarking scheme inaccordance with the invention will be described as a system forattaching invisible labels to video contents but the teachings canobviously be applied to any other contents, including audio andmultimedia. We will hereinafter often refer to this method as JAWS (JustAnother Watermarking System).

[0018]FIG. 1 shows a practical embodiment of the watermark embedder toprovide background information. The embedder comprises an image source11 which generates an image P, and an adder 12 which adds a watermark Wto the image P. The watermark W is a noise pattern having the same sizeas the image, e.g. N₁ pixels horizontally and N₂ pixels vertically. Thewatermark W represents a key K, i.e. a multi-bit code which is to beretrieved at the receiving end.

[0019] To avoid that the watermark detection process needs to search thewatermark W over the large N₁×N₂ space, the watermark is generated byrepeating, and if necessary truncating, smaller units called “tiles”W(K) over the extent of the image. This “tiling” operation (15) isillustrated in FIG. 2. The tiles W(K) have a fixed size M×M. The tilesize M should not be too small: smaller M implies more symmetry in W(K)and therefore a larger security risk. On the other hand M should not betoo large: a large value of M implies a large search space for thedetector and therefore a large complexity. In JAWS we have chosen M=128as a reasonable compromise.

[0020] Then, a local depth map or visibility mask λ(P) is computed (16).At each pixel position, λ(P) provides a measure for the visibility ofadditive noise. The map λ(P) is constructed to have an average valueequal to 1. The extended sequence W(K) is subsequently modulated (17)with λ(P), i.e. the value of the tiled watermark W(K) at each positionis multiplied by the visibility value of λ(P) at that position. Theresulting noise sequence W(K,P) is therefore dependent on both the key Kand the image content of P. We refer to W(K,P) as an adaptive watermarkas it adapts to the image P.

[0021] Finally, the strength of the final watermark is determined by aglobal depth parameter d which provides a global scaling (18) of W(K,P).A large value of d corresponds to a robust but possibly visiblewatermark. A small value corresponds to an almost imperceptible but weakwatermark. The actual choice of d will be a compromise between therobustness and perceptibility requirements. The watermarked image Q isobtained by adding (12) W=d×W(K,P) to P, rounding to integer pixelvalues and clipping to the allowed pixel value range.

[0022] In order to embed the multi-bit code K in the watermark W, everytile W(K) is built up from a limited set of uncorrelated basic orprimitive tiles {W₁ . . . W_(n)} and shifted versions thereof, inaccordance with${W(K)} = {\sum\limits_{i,j}{s_{i_{j}}{{shift}\left( {W_{i},k_{i_{j}}} \right)}}}$

[0023] where “shift (W_(i), k_(i) _(j) )” represents a spatial shift ofa basic M*M tile W_(i) over a vector k_(i) _(j) with cyclic wrap around.The signs sε{−1,+1 } and the shifts k depend on the key K via anencoding function E (13). It is the task of the detector to reconstructK after retrieving the signs s_(i) and the shifts k_(i). Note that eachbasic tile WI may occur several times. In FIG. 1, the encoder 13generates W(K)=W₁+W₂-W₂′ where W₂′ is a shifted version of W₂. FIG. 3illustrates this operation.

[0024]FIG. 4 shows a schematic diagram of a watermark detector. Thewatermark detector receives possibly watermarked images Q. Watermarkdetection in JAWS is not done for every single frame, but for groups offrames. By accumulating (21) a number of frames the statistics ofdetection is improved and therefore also the reliability of detection.The accumulated frames are subsequently partitioned (22) into blocks ofsize M×M (M=128) and all the blocks are stacked (23) in a buffer q ofsize M×M. This operation is known as folding. FIG. 5 illustrates thisoperation of folding.

[0025] The next step in the detection process is to assert the presencein buffer q of a particular noise pattern. To detect whether or not thebuffer q includes a particular watermark pattern W, the buffer contentsand said watermark pattern are subjected to correlation. Computing thecorrelation of a suspect information signal q with a watermark pattern wcomprises computing the inner product d=<q,w> of the information signalvalues and the corresponding values of the watermark pattern. For aone-dimensional information signal q={q_(n)} and watermark patternw={w_(n)}, this can be written in mathematical notation as:${d = {\frac{1}{N}{\sum\limits_{n = 1}^{N}{q_{n}w_{n}}}}},$

[0026] For the two-dimensional M×M image q={q_(ij)} and watermarkpattern W={w_(ij)}, the inner product is:$d = {\frac{1}{M^{2}}{\sum\limits_{i = 1}^{M}{\sum\limits_{j = 1}^{M}{q_{ij}{w_{ij}.}}}}}$

[0027] In principle, the vector k_(i) by which a tile W_(i) has beenshifted can be found by successively applying W_(i) with differentvectors k to the detector, and determining for which k the correlationis maximal. However, this brute force searching algorithm is timeconsuming. Moreover, the image Q may have undergone various forms ofprocessing (such as translation or cropping) prior to the watermarkdetection, so that the detector does not know the spatial location ofthe basic watermark pattern W_(i) with respect to the image Q.

[0028] Instead of brute force searching JAWS exploits the structure ofthe patterns W(K). The buffer q is examined for the presence of theseprimitive patterns, their signs and shifts. The correlation d_(k) of animage q and a primitive pattern w being shifted by a vector k (k_(x)pixels horizontally and k_(y) pixels vertically is:$d_{k} = {\frac{1}{M^{2}}{\sum\limits_{i = 1}^{M}{\sum\limits_{j = 1}^{M}{q_{ij}{w_{{i + k_{x}},{j + k_{y}}}.}}}}}$

[0029] The correlation values d_(k) for all possible shift vectors k ofa basic pattern W_(i) are simultaneously computed using the Fast Fouriertransform. As shown in FIG. 4, both the contents of buffer q and thebasic watermark pattern W_(i) are subjected to a Fast Fourier Transform(FFT) in transform circuits 24 and 25, respectively. These operationsyield:

{circumflex over (q)}=FFT(q) and

ŵ=FFT(w),

[0030] where {circumflex over (q)} and ŵ are sets of complex numbers.

[0031] Computing the correlation is similar to computing the convolutionof q and the conjugate of W_(i). In the transform domain, thiscorresponds to:

{circumflex over (d)}={circumflex over (q)}{circle over (×)}conj(ŵ)

[0032] where the symbol {circle over (×)} denotes pointwisemultiplication and conj( ) denotes inverting the sign of the imaginarypart of the argument. In FIG. 4, the conjugation of ŵ is carried out bya conjugation circuit 26, and the pointwise multiplication is carriedout by a multiplier 27. The set of correlation values d={d_(k)} is nowobtained by inverse Fourier transforming the result of saidmultiplication:

d=IFFT(d)

[0033] which is carried out in FIG. 4 by an inverse FFT circuit 28. Thewatermark pattern W_(i) is detected to be present if a correlation valued_(k) is larger than a given threshold.

[0034]FIG. 6A shows a graph of correlation values d_(k) if the presenceof watermark pattern W₁ (see FIGS. 1 and 3) in image Q is being checked.The peak 61 indicates that W₁ is indeed found. The position (0,0) ofthis peak indicates that the pattern W₁ applied to the detector happensto have the same spatial position with respect to the image Q as thepattern W₁ applied to the embedder. FIG. 6B shows the graph ofcorrelation values if watermark pattern W₂ is applied to the detector.Two peaks are now found. The positive peak 62 at (0,0) denotes thepresence of watermark W₂, the negative peak 63 at (48,80) denotes thepresence of watermark—W₂′. The relative position of the latter peak 63with respect to peak 62 (or, what is similar, peak 61) reveals therelative position (in pixels) of W₂′ with respect to W₂, i.e. the shiftvector k. The embedded data K is derived from the vectors thus found.

[0035] The embedded information may identify, for example, thecopy-right holder or a description of the content. In DVDcopy-protection, it allows material to be labeled as ‘copy once’, ‘nevercopy’, ‘no restriction’, ‘copy no more’, etc. FIG. 7 shows a DVD drivefor playing back an MPEG bitstream which is recorded on a disc 71. Therecorded signal is applied to an output terminal 73 via a switch 72. Theoutput terminal is connected to an external MPEG decoder and displaydevice (not shown). It is assumed that the DVD drive may not play backvideo signals with a predetermined embedded watermark, unless otherconditions are fulfilled which are not relevant to the invention. Forexample, watermarked signals may only be played back if the disc 71includes a given “wobble” key. In order to detect the watermark, the DVDdrive comprises a watermark detector 74 as described above. The detectorreceives the recorded signal and controls the switch 72 in response towhether or not the watermark is detected.

[0036] The Fourier coefficients {circumflex over (d)} are complexnumbers, that is, they have a real part and an imaginary part, or amagnitude and a phase. The inventors have found that the reliability ofthe detector is significantly improved if the magnitude information isthrown away and the phase is considered only. FIG. 8 shows an embodimentof the detector's correlation circuitry in accordance with theinvention. The embodiment differs from the one shown in FIG. 4 in that amagnitude normalization circuit 30 has been inserted between themultiplier 27 and the inverse Fourier Transform circuit 28. Theoperation of the normalization circuit comprises pointwise dividing eachcoefficient by its magnitude. In mathematical notation:

{circumflex over (d)}:={circumflex over (d)}Φabs(d)  (1)

[0037] where Φ denotes pointwise division and abs( ) denotes:

abs({circumflex over (d)} _(k))={square root}{square root over(R({circumflex over (d)})} _(k))² +I({circumflex over (d)} _(k))²  (2)

[0038] where R( ) and I( ) denote the real and imaginary part of theargument, respectively.

[0039] Said normalization of magnitudes is referred to as SymmetricalPhase Only Matched Filtering (SPOMF). FIGS. 9A and 9B illustrate theeffect of SPOMF correlation. More particularly, FIG. 9A shows thecorrelation values d_(k) when using linear correlation, i.e. without themagnitude normalization circuit 30. The correlation value d₀₀, expressedin units of standard deviation of the whole matrix, amounts to 9.79.FIG. 9B shows the correlation values when using SPOMF correlation. Thecorrelation value d₀₀ is now 62.77 times the standard deviation. It willbe appreciated that the peak in FIG. 9B can more reliably be detectedthan the peak in FIG. 9A.

[0040] Because normalizing the magnitudes of {circumflex over (d)} isequivalent to normalizing the magnitudes of both {circumflex over (q)}and ŵ, the normalization circuit 30 in FIG. 8 may be replaced by twonormalization circuits after the FFT circuits 24 and 25. However, theembedded watermark will already have a reasonably white (flat) frequencyspectrum because it is a pseudo-random noise pattern in practice, inwhich each sample is independently and identically drawn from a normaldistribution. In view hereof, normalizing the magnitude of theinformation signal only has been found to suffice. FIG. 10 shows such anembodiment. The magnitude normalization circuit 30 is now locatedbetween the FFT circuit 24 and the multiplier 27. In this embodiment,the magnitudes of {circumflex over (d)} are not exactly, butsubstantially, the same.

[0041] It should further be noted that the FFT and the conjugation ofthe applied watermark W_(i) (c.f. circuits 25 and 26, respectively, inFIGS. 4, 8 and 10), as well as the optional normalization of themagnitudes of ŵ, can be pre-computed and stored in a memory.

[0042] The inventors have further found that the detection strength doesnot decrease significantly if a more course approximation for thenormalization is used. Instead of dividing each Fourier coefficient byits magnitude as defined by equations (1) and (2), it suffices to divideeach coefficient by its real or imaginary part, depending on which isthe largest. The magnitudes will then vary between 1 and {squareroot}{square root over (2)}, the detection strength decreasing by 1%only. The advantage is saving of computational complexity, because thesquare root in equation (2) needs not be calculated. In such anembodiment, the normalization circuit 30 performs the followingalgorithm:

if R({circumflex over (d)} _(k))≧^(I)({circumflex over (d)} _(k))

then {circumflex over (d)} _(k) :={circumflex over (d)} _(k)/R({circumflex over (d)} _(k))

else {circumflex over (d)} _(k) :={circumflex over (d)} _(k)/I({circumflex over (d)} _(k))

[0043] where {circumflex over (d)}_(k) is a complex Fourier coefficientof {circumflex over (d)} (or {circumflex over (q)} or ŵ as the case maybe), R({circumflex over (d)}_(k)) and I({circumflex over (d)}_(k)) arethe real and imaginary part of {circumflex over (d)}_(k), respectively,and/denotes division.

[0044] A further reduction of complexity can be obtained if thenormalization includes divisions by only powers of 2. The magnitudeswill then vary between 1 and {square root}{square root over (2)}, thedetection strength decreasing by 2.5%. Division by powers of 2 isparticularly efficient in most common floating point number systems inwhich the real and imaginary parts of complex coefficients arerepresented as:

sign·mantissa·2^(exp)

[0045] where mantissa is a fixed point representation of a numberbetween 1 and 2. The above normalization can now be simplified asfollows:

if exp(R({circumflex over (d)} _(k)))≧exp(I({circumflex over (d)} _(k)))

then divider:=exp(R({circumflex over (d)} _(k)))

else divider:=exp(I({circumflex over (d)} _(k)))

exp(R({circumflex over (d)} _(k))):=exp(R({circumflex over (d)}_(k)))−divider

exp(I({circumflex over (d)} _(k))):=exp(I({circumflex over (d)}_(k)))−divider

[0046] In summary, an improved method and arrangement for detecting awatermark in an information (e.g. image or video) signal are disclosed.The detection is more reliable and less vulnerable to image processingby subjecting the suspect image (q) and the watermark to be detected(W_(i)) to Symmetrical Phase Only Matched Filtering (24-28,30) prior todetecting (29) the amount of correlation (d) between said signals.

1. A method of detecting a watermark in an information signal, comprising the steps of: computing a value indicative of the correlation of said information signal and said watermark; and detecting whether said correlation value is larger than a predetermined threshold; characterized in that said computing comprises the steps of Fourier transforming the information signal and the watermark into respective Fourier coefficients, multiplying corresponding coefficients to obtain Fourier coefficients representing correlation values each corresponding with one of a plurality of positions of the watermark with respect to the information signal, and inverse transforming the Fourier coefficients representing the correlation values; said step of detecting comprises detecting whether at least one of said correlation values is larger than the predetermined threshold; the method further including the step of normalizing the Fourier coefficients representing the correlation values to substantially the same magnitudes.
 2. A method as claimed in claim 1, wherein said step of normalizing is applied to the Fourier coefficients representing the information signal.
 3. A method as claimed in claim 2, wherein said step of normalizing is further applied to the Fourier coefficients representing the watermark.
 4. A method as claimed in claim 1, wherein said step of normalizing includes dividing each Fourier coefficient by its respective magnitude.
 5. A method as claimed in claim 1, wherein said step of normalizing includes dividing each Fourier coefficient by its real or imaginary part depending on which is the largest.
 6. A method as claimed in claim 5, wherein said dividing comprises dividing by a power of
 2. 7. An arrangement for detecting a watermark (w) in an information signal (q), comprising: means (24-28) for computing a value (d) indicative of the correlation of said information signal and said watermark; and means (29) for detecting whether said correlation value is larger than a predetermined threshold; characterized in that said computing means comprise means (24,25) for Fourier transforming the information signal and the watermark into respective Fourier coefficients, means (27) for multiplying corresponding coefficients to obtain Fourier coefficients representing correlation values each corresponding with one of a plurality of positions of the watermark with respect to the information signal, and means (28) for inverse transforming the Fourier coefficients representing the correlation values; said detecting means (29) are arranged to detect whether at least one of said correlation values is larger than the predetermined threshold; the arrangement further including means (30) for normalizing the Fourier coefficients representing the correlation values to substantially the same magnitudes.
 8. A device for recording and/or playing back an information signal, comprising means (72) for disabling recording and/or playback of the video signal in dependence upon the presence of a watermark in said signal, characterized in that the device comprises an arrangement (74) for detecting said watermark in the signal as claimed in claim
 7. 